\section{Filters in camlp4}


\subsection{Map Filter}
\label{transform}


The filter \emph{Camlp4MapGenerator} reads \emph{OCaml} type
definitions and generate a class that implements a map traversal.  The
generated class have a method \textit{per type} you can override to
implement a \textit{map traversal}. It needs to read the
\textit{definition} of the type.

Camlp4 uses the \textbf{filter} iteself to bootstrap.


\begin{ocamlcode}
(** file Camlp4Ast.mlast *)
class map = Camlp4MapGenerator.generated;
class fold = Camlp4FoldGenerator.generated;
\end{ocamlcode}

As above, \verb|Camlp4.PreCast.Ast| has a corresponding map traversal
object, which could be used by you: (the class was generated by our
filter) \verb|Ast.map| is a class

\begin{ocamlcode}
let b = new Camlp4.PreCast.Ast.map ;;
val b : Camlp4.PreCast.Ast.map = <obj>
\end{ocamlcode}



\subsection{Filter Examples}
Using filters is a declarative way of doing ast transformation, the
input is a legal syntax construct(not necessarily leagal ocaml ast),
the output is a legal ocaml AST.

\subsubsection{Example: Map Filter}
You can also generate map traversal for ocaml type. \textit{put your
  type definition before} you macro, as Listing \ref{Map Filter
  Example}

\inputminted[fontsize=\scriptsize,bgcolor=lightlightgray]
{ocaml}{code/camlp4/filters/ast_map/ast_map.ml}
\captionof{listing}{Map Filter Example \label{Map Filter Example}}

Without filter, you would write the transformer by hand like this
\inputminted[fontsize=\scriptsize, fontsize=\scriptsize,
]{ocaml}{code/camlp4/filters/ast_map/ast_map_o.ml}
\captionof{listing}{Map Filter by Hand \label{Map Filter By Hand}}

Camlp4 use the filter in \verb|antiquot_expander|, for example, in
\textit{Camlp4Parsers/Camlp4QuotationCommon.ml} \ref{Quotation
  Expander}, in the definition of \verb|add_quotation|.

Using Register Filter has some limitations, like it first parse the
whole file, and then transform each \textit{structure item} one by one,
so the code generated before will \textit{not have} an effect on the
later code. This is probably what not you want.


\subsubsection{Example: Add Zero}
\inputminted[fontsize=\scriptsize,lastline=10,bgcolor=lightlightgray]
{ocaml}{code/camlp4/filters/add_zero/ast_add_zero.ml}
\captionof{listing}{Ast Filter Add Zero\label{Ast Filter Add Zero}}

It's actually \textit{a nested pattern match} as follows
\inputminted[fontsize=\scriptsize, firstline=14,lastline=33]{ocaml}
{code/camlp4/filters/add_zero/ast_add_zero.ml}
To make life easier, you can write like this 

In the module \verb|Camlp4.PreCast.AstFilters|, which is generated by
\verb|Camlp4.Struct.AstFilters.Make|[\ref{lst:AstFilters}], there are
some utilities to do filtering over the ast. It's actually very simple.

\begin{ocamlcode}
    type 'a filter = 'a -> 'a
    val register_sig_item_filter : Ast.sig_item filter -> unit
    val register_str_item_filter : Ast.str_item filter -> unit
    val register_topphrase_filter : Ast.str_item filter -> unit
    val fold_interf_filters : ('a -> Ast.sig_item filter -> 'a) -> 'a -> 'a
    val fold_implem_filters : ('a -> Ast.str_item filter -> 'a) -> 'a -> 'a
    val fold_topphrase_filters :
      ('a -> Ast.str_item filter -> 'a) -> 'a -> 'a
\end{ocamlcode}
\captionof{listing}{Filters module \label{Filters module}}


\subsubsection{Fold filter}
\begin{ocamlcode}
  class x = Camlp4FoldGenerator.generated ;
\end{ocamlcode}

\subsubsection{Meta filter}
\todo{insert my roll own filter}
Meta filter needs a module name, however, it also needs the source of
the definition of the module. (Since OCaml does not have type
reflection). There are some problems here, first you want to separate
the type definition in another file, this could be achieved while
using macro \verb|INCLUDE|, and \verb|Camlp4Trash|, however, you also
want to make your type definition to using another syntax extension,
i.e, \verb|sexplib|, \verb|deriving|, \verb|deriving| will
\textit{generate a bunch of modules and types}, which conflicts with
our \verb|Meta filter|. So, be careful here.


However, the problem can not be solved for meta filter, it can be
solved for \verb|map, fold| filter, for the meta filter, it will
introduce the name of \verb|Camlp4Trash| into the source tree, so you
can not trash the module otherwise it will not compile. The only way
to do it is to write your own \verb|TrashModule| ast filter.

Fortunately, it's not hard to roll your own filter.
First 
\inputminted[fontsize=\scriptsize, fontsize=\scriptsize]{ocaml}{camlp4/code/jake/json_ast.ml}
\inputminted[fontsize=\scriptsize, fontsize=\scriptsize]{ocaml}{camlp4/code/jake/pa_json_ast.ml}
Notice that we have \verb|Camlp4TrashX|, you can not trash it, due to
the fact that the generated code needs it.

\subsubsection{Lift filter}

These functions are what \emph{Camlp4AstLifter uses} to lift the AST,
and also how \emph{quotations are implemented }
A example of meta filter could be found here .
Here we do a interesting experiment, lift a ast for serveral times

By the \textit{lift filter} you can see its \textbf{internal
  representation}, textual code does not gurantee its correctness, but
the AST representation could gurantee its correctness.


\begin{bashcode}
camlp4o -filter lift -filter lift  test_lift.ml -printer o > test_lift_1.ml
camlp4o -filter lift -filter lift  test_lift_1.ml -printer o > test_lift_2.ml
\end{bashcode}
You can lift it several times to see it grows exponentially


\subsubsection{Macro Filter}

\inputminted[fontsize=\scriptsize]{ocaml}
{code/camlp4/filters/macros/first.ml}
\captionof{listing}{Example Macro\label{Example Macro}}

\subsection{Example Tuple Map}
\inputminted[fontsize=\scriptsize,lastline=35]{ocaml}
{code/camlp4/filters/map/tuple_map.ml}
\captionof{listing}{Example Tuple Map \label{Example Tuple Map}}

\subsection{Location Strip filter}
Replace location with \verb|Loc.ghost|


Might be useful when you compare two asts? YES!  idea? how to use
lifter at toplevel, how to beautify our code, without the horribling
output? (I mean, the qualified name is horrible, but you can solve it
by open the Module)

\subsubsection{Camlp4Profiler}
Inserts profiling code


\subsubsection{Camlp4TrashRemover}
\inputminted[fontsize=\scriptsize, fontsize=\scriptsize, lastline=40]{ocaml}{camlp4/code/jake/pa_trashmover.ml}

\subsubsection{Camlp4ExceptionTracer}




\subsection{Linking Problem}
You syntax extension may depends on other modules, make sure your
\textit{pa\_xx.cma} contains all the modules statically. You can write
a \textit{pa\_xx.mllib}, or link the module to \textit{cma} file by hand.

For instance, you \textit{pa\_filter.cma} depends on \textit{Util}, then
you can do some stuff as follows

\begin{bashcode}
  ocalmc -a pa_filter.cmo util.cmo -o pa_filer.cma
  camlp4o -parser pa_filter.cma  
\end{bashcode}

If you write \textit{pa\_xx.mllib} file, it would be something like

\begin{bashcode}
pa_filter
util
\end{bashcode}
If you want to \textit{use other libraries to write syntax extension}, make
sure you link \textit{all} libraries, including recursive dependency,
i.e, the require field of batteries.

\begin{bashcode}
ocamlc -a  -I +num -I `ocamlfind query batteries` nums.cma unix.cma
bigarray.cma str.cma batteries.cma pa_filter.cma -o x.cma
\end{bashcode}

You must link all the libraries \textit{recursively}, even you don't
need it at all. This is the \textit{defect} of the OCaml compiler.
\textit{-linkall} here links submodules, recursive linking needs you say
it clearly, you can find some help in the \textit{META} file.

We can also test our filter seriously as follows

\begin{bashcode}
camlp4of -parser _build/filter.cmo filter_test.ml -filter lift -printer o   
\end{bashcode}

